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METHOD AND DEVICE FOR RETRIEVING CUSTOMIZED MOVING IMAGE DATA 



retrieving customized moving image data (i.e. videos, movies, 
animations, etc., hereinafter referred to as video data) from a 
remote source and a moving image data source for remote 
retrieval. The video data may comprise relatively short video 

10 clips, animations or entire video films (movies), for instance 
in the case of video-on-demand. 

It is known how to provide digital video data over a 
network for access by remote users. However, previous systems 
have a number of drawbacks. In particular, video data files 

15 are typically very large and, hence, take a considerable amount 
of time to download over a network such as the Internet. Their 
large size may also make them inconvenient for use with the 
small processor memories or limited data storage capacities of 
many user terminals. Furthermore, video data files are often 

20 provided on network servers in a format which may be 

inappropriate for many user terminals or for the intended end 
use. 

According to the present invention, there is provided a 
method of retrieving customized video data, the method 
25 comprising*: 



FROM A REMOTE SOURCE 



AND A MOVING IMAGE DATA SOURCE FOR REMOTE RETRIEVAL 



5 



The present invention relates to a method and device for 



30 



35 



(a) 



(b) 



(c) 



(d) 



specifying a particular video data file and 
requesting an image of video data and information 
concerning the video data; 

retrieving from the source the image of the. video 
data and the information concerning the video data; 
specifying parameters determining at least one of the 
desired spatial, temporal and format characteristics 
of the customized video data and sending the 
parameters to the source; and 

receiving video data transmitted by the source after 
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customization on the basis of the specified 
parameters . 

According to the present invention, there is also provided 
a device for retrieving customized video data from a remote 
5 source, the device including: 

(a) means for specifying a particular video data file and 
requesting an image of video data and information 
concerning the video data; 

(b) means for receiving from the remote source the image 
10 of the video data and the information concerning the 

video data; 

(c) means for transmitting parameters determining at 
least one of the desired spatial, temporal and format 
characteristics of the video data and for requesting 

15 customization of the video data on the basis of the 

specified customization parameters; and 

(d) means for retrieving video data supplied from the 
source after customization on the basis of the 
specified customization parameters. 

20 In conjunction with this method, there is also provided a 

video data source for remote access, the source: 

(a) being responsive to a remote user's selection of a 
particular video data file to prepare an image of 
video data and information concerning the video data, 
25 (b) transmitting to the user the image of the video data 

and the information concerning the video data, 

(c) being responsive to parameters determining at least 
one of the desired spatial, temporal and format 
characteristics of the video data received from the 

30 remote user, and 

(d) transmitting to the user video data from the original 
video data file after customization on the basis of 
the received parameters. 

Once a user terminal has accessed the source, the source 
35 can transmit to the user a control program to enable the user 
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to specify the customization parameters. The control program 
may take the form of a Java applet. In this way, the user 
terminal does not in itself require any features dedicated to 
the remote source, but acquires the ability to control the 
source upon accessing it. The source can be used by several 
remote users concurrently. 

The specifying parameters may specify merely default or 
unmodified video data or may specify customization of the video 
data. Furthermore, the specifying parameters may additionally 
determine one or both of auditory and visual characteristics. 

The customized video data may include one or more of the 
following characteristics : 



(i) one or more temporal extracts from the original 
video data or the full original length, 

(ii) one or more spatial extracts from the original 
video data or the full original frame size, 

(iii) a magnification or demagnif ication (zooming or 
dezooming) of the original video data or the 
unmodified original dimensions, 

(iv) a time lapse ratio relative to the original 
video data where this value may be unity, 

(v) a selection of one or more of the soundtracks 
from among those accompanying the original video data 
or the omission of soundtrack data, 

(vi) a modification of the visual appearance of the 
original video data by the application of selected 
image processing procedures or no change to the 
visual appearance, 

(vii) encoding of the video data using a selected 
video format that may be the same as that of the 
original video data, 

(viii) encoding of the video data using a specified 
frame rate that may be the same as that of the 
original video data, 

(ix) encoding of the video data using a selected 
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compression/decompression algorithm (codec) that may 
be the same as that of the original video data, and 
(x) encoding of the video data using a selected 
compression quality setting that may be the same as 
5 that of the original video data. 

The present invention permits the user to customize a 
personal version of a particular video data file available on 
a network server or on an Internet server prior to downloading 
it, hence permitting the video to be adapted to suit the 
10 bandwidth of its network connection, the available data storage 
capacity, the video playback software available on the user's 
computer, or the subsequent usage to which the video will be 
put . 

Specifically, the present invention makes it possible to 

15 edit a selected video data file spatially and temporally, to 
modify the magnification and the time lapse ratio of the 
customized version, to select the soundtracks, to modify the 
visual appearance of the original video data, and to change the 
video format, the frame rate, and the nature and quality of the 

20 compression applied to the customized video data, prior to 
downloading to the user. In this way, the video quality and 
playback characteristics can be matched to the intended use to 
which the video will be put, and substantial savings in 
download time and data storage requirement may be obtained. 

25 This may be advantageous for the user. 

For example, in some cases a user may require only part of 
the full video temporal sequence, perhaps only the first half, 
or may require only part of each video frame, for instance only 
the bottom right quadrant. By specifying such customization 

30 parameters to the source, which modifies the video data in 

response to them before transmission to the user, unnecessary 
data is not transmitted over the network. In consequence, both 
the transfer (downloading) time and the video data file size 
may be reduced, the latter thus requiring less data storage 

35 space on the user terminal. This is particularly useful if the 
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user terminal has a small processor memory and/or limited data 
storage capacity. 

Rather than downloading an entire video data file and then 
edit and process it locally, a user merely requests the 
5 customization of the video data required, leaving the 

computational burden for the video editing and customization 
processes to the source, such that a relatively unsophisticated 
remote user terminal with limited processing power can obtain a 
video data file edited as required, without the need for any 

10 local video editing capabilities. 

Thus, using the present invention, a user can download 
only that part of a video which is of interest, either in 
temporal or spatial terms. Similarly, the user can download 
the video data with the image quality or format to suit the 

15 user terminal or application. 

If, for example, the viewer wishes to download and view an 
entire video for which a version in streaming format is not 
available, but has only a low-speed network connection and 
limited data storage capacity, a highly compressed and 

20 spatially demagnified (dezoomed) version might be chosen to 

enable rapid Internet transmission and uninterrupted viewing at 
full frame rate. 

Alternatively, if the video is to be downloaded for 
subsequent local projection, the projectionist will wish to 

25 ensure that it is downloaded in full length and at full frame 
size, with the highest possible quality compatible with the 
real-time video display bandwidth of the local video server, 
even if this means a prolonged downloading processes that may 
be conveniently performed overnight. 

30 If the purpose is instead to undertake frame-by- frame 

image analysis of a few particular events, in order to quantify 
particular features in the images, then it may be desirable to 
download only particular areas of interest within selected 
temporal video clips from the original video data file, but to 

35 do so at the highest possible resolution. 
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The present invention will be more clearly understood from 
the following description, given by way of example only, with 
reference to the accompanying drawings, in which: 

Figure 1 illustrates schematically an embodiment of the 
5 present invention on a network; 

Figure 2 illustrates schematically an alternative 
embodiment of the invention on a network; 

Figure 3 illustrates the control panel of a user terminal 
operating according to an embodiment of the present invention; 
10 and 

Figure 4 illustrates another control panel of a user 
terminal operating according to an embodiment of the present 
invention. 

As is well known and as is illustrated in Figures 1 and 2, 
15 a number of user terminals 2,4,6 may communicate with a server 
8 over a network 10. In particular, it is possible to provide 
the user terminals 2,4,6 and the server 8 with connections to 
the Internet 10, such that they may communicate over the 
Internet 10. 

20 The server 8 may provide many different services to the 

user terminals 2,4,6. One very common use of a server 8 is to 
provide information. The information provided may include many 
different types of data, including text, sounds and images. 
The present application is particularly concerned with the 

25 provision of digitized moving image data (i.e. videos, movies, 
animations, etc., herein referred to as video data). The 
server 8 may store the video data itself or may handle video 
data stored on another server at a separate location 12 (Figure 
2) . In the latter case, the video data may be transferred to 

30 the server 8 from another server at a separate location 12 by a 
dedicated line or, as illustrated in Figure 2, via a network or 
the Internet 10. 

It is now proposed to provide a system based on the server 
8 and the user terminals 2,4,6 which enables editing and 

35 customization of specified video data files to be undertaken on 
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the server 8 by remote users, and for the resulting customized 
video data file to be downloaded from the server 8 to the user 
terminals 2,4, 6. In particular, the system is intended to 
allow only desired spatio-temporal portions of particular video 
5 files to be downloaded, and to allow the edited video data to 
be downloaded with a specified magnification, time lapse ratio 
and visual appearance, with selected soundtracks, and with a 
specified video format, frame rate, codec and compression 
quality setting. 

10 The preferred embodiment to be described below is called 

VIDOS, since the present invention in a VJdeo Download 
Specification system. The preferred embodiment described as 
VIDOS may be written as a platform-independent Java server 
program running on a VIDOS server, together with Java applets 

15 running on the user terminals by which the users can select the 
required video data files and specify the parameters to be used 
in preparing the customized versions of these files. Although 
other methods of implementing the invention are possible, Java 
is a form of programming that is particularly well suited for 

20 interactive work over a network or the Internet* The server 8 
preferably stores the Java server program and Java user applets 
relating to the video selection and customization. When one of 
the user terminals 2,4,6 accesses the VIDOS server 8, the 
appropriate Java applet is sent to the particular user terminal 

25 2,4, 6 and this enables the user terminal 2,4,6 to send 

appropriate parameters to the server 8, first to specify the 
selected video data file and subsequently to customize the 
video data prior to downloading. More specifically, it 
permits the user to enter the parameters required to customize 

30 a personal edited version of a particular video data file, to 
be informed of the consequences of these parameter choices in 
terms of the compression time and final video file size, and to 
see a preview of the first 50 frames of the customized version 
prior to final downloading of the complete customized video. 
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On-line help may be available to guide the naive user in making 
these choices. All the parameters have initial default values, 
and, in case of incompatibilities in the choices made, the 
program may inform the user and offer appropriate alternatives. 
5 The video customization parameters may be passed over a 

network or the Internet, for example using standard World Wide 
Web protocols, to the main VIDOS server program running on the 
distant server. The server may operate using the Java program 
together with standard programming tools and libraries to 
10 produce the desired customized version of the video. It may 
then send a message to the user terminal, providing the user 
with access to the customized video data and permitting the 
user to initiate the customized video downloading process. 

The nature of the Java VIDOS server program is such that 
15 several user terminals 2,4,6 may undertake concurrent 
customizations of different video data files. 

The working of the present invention will now be described 
in greater detail. Upon first accessing the server 8, each 
user terminal 2,4,6 preferably presents to the user a control 
20 panel (the video selection interface) , such as is illustrated 
in Figure 3. The user is first asked to specify the name and 
location of the video data file to be edited, which may be 
stored on the VIDOS server 8 itself or on another server at a 
separate location 12. Once this file has been located and, if 
25 necessary, transferred to the VIDOS server 8, an image of the 
video data is prepared by the server and ' transmitted to the 
user terminal. This may take the form of a still image of the 
first frame of the chosen video data file, as in the example 
illustrated in Figure 3, or it may comprise downloaded or 
30 streamed moving image data of reduced preview resolution, with 
appropriate viewing controls. The server preferably also 
transmits to the user terminal textual data relating to the 
size and format of the full video data file as stored by the 
server 8 or obtained from another server at a separate location 
35 12. The user may at this stage choose simply to download the 
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entire video file in its original format, or alternatively may 
choose to make use of the VIDOS system to create a customized 
version of the original video data prior to downloading it. 
Having selected use of the VIDOS video download 
5 specification system, the user terminal presents the user with 
a second control panel (the video customization interface) , 
such as is illustrated in Figure 4. Once again, an image 
representative of the video data is displayed in the upper 
portion of the control panel, and controls are provided whereby 

10 the user is able to enter customization parameters. 

In entering the customization parameters, the user can 
first determine the spatial extent of each video frame to be 
downloaded. By default, this is the full frame, but, as shown 
in the example (Figure 4), the user can interactively limit the 

15 spatial region of each frame to a particular rectangular area 
of interest (Aol) . The dimensions of this may be defined in a 
number of ways: for example by using mouse clicks on the 
displayed image of the video data to identify the upper left 
and lower right corners of the Aol box; by typing the cropping 

20 values in the four text boxes (left, right, top and bottom) 
immediately below the video image; or by increasing or 
decreasing the values in these boxes using the adjacent + and - 
spinner buttons. In whichever way the user chooses to specify 
the Aol, the selected area is preferably displayed by an 

25 outline on the illustrated image of the video data (Fig. 4) . 

It would alternatively be possible to select more than one area 
of interest, and to position several areas of interest in 
different ways in the final video frames. 

A magnification/ demagnif ication (zoom/dezoom) parameter 

30 may also be specified. In the illustrated control panel 

(Figure 4), this is the right-hand box on the top row. Taking 
account of this zoom factor, the cropped original frame size 
and the resulting download frame size are interactively 
displayed in the adjacent area. In addition, the file size of 

35 the output video after such spatial editing and an estimate of 
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the conversion time required to generate the customized video 
may be indicated (as is illustrated near the bottom of Figure 
4)* These values may also be interactively updated as the Aol 
cropping parameters and the other download customization 
5 parameters described below are modified. 

As illustrated in Figure 4,^ in addition to the controls 
specifying the spatial constraints to be imposed on the 
original video data prior to downloading, it is also possible 
to provide similar controls by which the user can undertake 
10 temporal editing of the original video data. The default 
values include the whole of the original video. The system 
ss . preferably enables the user to select one or more temporal 

iQ sections (video clips) from the original video data, and to 

\1 arrange such video clips, if more than one, in a specified 

15 sequence in the final customized video. In the example shown 
;;j in Figure 4, by using controls to enter the first and last 

*- frame numbers, the user is able to specify the time points in 

~j the original video that will become the beginning and end of 

^ the customized video. The system preferably enables the user 

^ 20 to view the frames to be chosen for the start and end times of 
=3 each video clip, for example by positioning sliders on a video 

time-line display control. 

The system preferably enables the user to specify a time 
lapse ratio between the original and the customized video data, 
25 which if other than unity enabling the user to view events 

either in slow motion or speeded up relative to the original. 

If the original video has a sound track, a control is 
preferably provided to make it possible to specify whether or 
not this should be included in the customized video. . Such 
30 selection can be extended to the choice of one or more 

particular soundtracks from several original soundtracks, for 
example multiple language soundtracks, or the omission of the 
soundtrack data altogether. 

One can also preferably alter the visual appearance of the 
35 original video data by applying simple image processing 
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procedures such as contrast enhancement, smoothing or 
sharpening, pseudocolouring, etc. 

Once the spatial, temporal, auditory and visual parameters 
have been specified, the user may then use further controls to 
5 specify other customizing parameters. As illustrated, drop- 
down list boxes are provided to specify the customizing 
parameters that relate to the desired video encoding. The 
first of these preferably specifies the video format to be 
used, with a choice between several commonly-used industry 
10 standard digital video formats such as AVI (Audio Video 

Interleave), QT (Quick Time) or MPEG-1. The second is for the 
q video frame rate, for example either 10, 15, 20, 25 or 30 

frames per second. Then comes the choice of the codec to be 
bj applied to the video, for example Cinepak, Indeo, Motion- JPEG, 

fll 15 MPEG-1 or other industry standard codecs, with these choices 
: J preferably being automatically adapted to the initial choice of 

j " = video format, as appropriate. The user can then specify the 

Q compression quality setting, for example choosing between 

^ ^Uncompressed' (not available for certain formats such as MPEG- 

20 1), x Low' , 'Medium' or 'High' (which may correspond to bit 

rates of 579,000, 1,158,000 and 2,316,000 bits per second for 
MPEG-1, or to values of 0.3, 0.6 and 0.9 for the spatial 
quality setting of other codecs) . 

As mentioned above, all of these parameters preferably 
25 have initial default values. If incompatible choices are made, 
the system preferably informs the user and offer appropriate 
alternatives. Every time a parameter choice is made, the 
displayed estimates of the final video data file size and 
conversion time can be automatically updated. These estimates 
30 may be determined from a data table derived from prior test 
video data conversions undertaken on the particular hardware 
upon which the VIDOS server program is running, under the full 
range of possible compression conditions, the actual conversion 
times obviously depending on the performance of the server on 
35 which the system is running and the number of concurrent users. 
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In the light of the feedback provided by the VIDOS server, 
the user may revise the input customization parameters. The 
system preferably provides a Preview * control for use once the 
user believes that the customization parameters have been 
5 optimized for the required purpose. As illustrated in Figure 
4, such a Preview button is provided at the foot of the VIDOS 
video customization interface screen. This preferably 
activates the customization of the first section of the video, 
for example the first fifty frames, which is then downloaded 

10 and displayed on the user terminal. In this way, the user is 
given the opportunity to make a final quality assurance check, 
to ensure that the customized video will have the required 
appearance and characteristics, before the customization 
parameters are applied to the entire video data file. At this 

15 stage, the customization parameters can be further revised, if 
desired, until the preview is acceptable. 

Finally, the system allows the user to activate the server 
to undertake the complete customization of the original video 
data according to the final customization parameters, and then 

20 to download the customized version of it to the user terminal. 
As illustrated, a Convert button may be provided at the foot of 
the VIDOS video customization interface screen. This may be 
clicked to activate the VIDOS server program to generate a full 
customized downloadable file with the precise specifications 

25 requested, containing just the required portions of the 
original video data in the desired format. The user may 
already know the time that this conversion is likely to take, 
having been advised of this at the foot of the VIDOS video 
customization interface screen. Depending upon the length of 

30 the original video, the nature of the customization requested, 
the computing power of the server, and the number of concurrent 
users, the conversion process take from a few seconds to 
several minutes. When this task is complete, the VIDOS server 
preferably sends a message to the user terminal confirming that 



WO 00/28741 



PCT/GB99/03751 



-13- 

the customization process has been completed, giving the 
compressed size of the customized version of the video, and 
providing a link to the customized video file itself, whereby 
the user may initiate the final downloading process, 
5 In conclusion, once a particular video data file has been 

identified on a network server or an Internet server, the 
present invention permits the user to edit and customize the 
video data from a distance over the network, without having 
first to download it in its entirety onto the user terminal, by 

10 specify the desired spatial, temporal, auditory, visual and 
format characteristics of the customized video data. One 
advantage of this is that the network-based video editing may 
be achieved without the need for the user to possess video 
editing software on the user terminal. By limiting the spatial 

15 or temporal extent of the video data to be downloaded, or by 
choosing a more efficient data compression system, the size of 
the customized video data file may be made very significantly 
smaller than that of the original video data. This capability 
may be used to improve the speed of network transfer of the 

20 video file, which is particularly important when working over 
low bandwidth network connections such as modems or satellite 
links. It may also be used to achieve substantial savings in 
data storage requirements on the user terminal, or to adapt the 
video data quality and format to suit the playback 

25 characteristics of the user terminal or the intended use to 
which the video will be put. 

The present invention has many potential uses. For 
instance, such network-based video editing would allow regional 
members of a TV network to download from a central video data 

30 repository particular portions of news items and other material 
relevant to their particular regions. Similarly, video clips 
may be selected from an archive and tailored for inclusion in 
live or computer-based professional training programs, for 
example in the nursing profession or for aircraft maintenance, 

35 and school teachers and university lecturers would be able to 
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select video segments, for example of particular twentieth 
century historical events from a national film archive, to 
illustrate their own classes and lectures. As the use of 
networked digital video databases increases, the ability to 
5 manipulate video data in this way will become ever more 
important. 



